Skip to content

fix: route datamate tools through local stdio mcp-engine instead of cloud#893

Draft
altimate-harness-bot[bot] wants to merge 3 commits into
mainfrom
fix/datamate-stdio-local-transport
Draft

fix: route datamate tools through local stdio mcp-engine instead of cloud#893
altimate-harness-bot[bot] wants to merge 3 commits into
mainfrom
fix/datamate-stdio-local-transport

Conversation

@altimate-harness-bot

@altimate-harness-bot altimate-harness-bot Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Problem

DatamateManagerTool.handleAdd() was creating MCP server entries pointing to the cloud MCP server (https://mcpserver.getaltimate.com) because buildMcpConfig() always fell back to DEFAULT_MCP_URL when mcpServerUrl was absent from altimate.json.

The cloud MCP runs in multi-user mode and reads connections from API headers — it has no access to ~/.altimate/connections.json. So any tool call like datamate-remote_jira_get_issue failed with "No connection configured for integration: jira" even when connections were configured locally.

Fix

datamate.tshandleAdd()

Replace buildMcpConfig() (HTTP remote) with a local stdio config:

const mcpConfig = {
  type: "local" as const,
  command: ["datamate", "start-stdio", "--datamate", args.datamate_id],
}

Each spawned datamate start-stdio process:

  • Reads connections.json from ~/.altimate/ directly (local engine, same as VS Code extension)
  • Discovers the right VS Code extension bridge at runtime via cwd-based sidecar matching in ~/.altimate/extension-rpc/
  • Multiple VS Code windows are handled correctly — each window's bridge writes a sidecar with its workspaceFolders; the spawned process matches on process.cwd() (set to project root by StdioClientTransport) and routes to the right bridge automatically

mcp-discover.tsstripSessionEnv()

When persisting a discovered server via mcp_discover, strip ALTIMATE_EXTENSION_RPC from the environment block before writing to disk. The socket path is session-specific (unique to the current extension host process) — hardcoding it would cause future sessions to connect to a dead or wrong bridge. Stripping it forces runtime discovery.

Prerequisites

Requires the global datamate CLI to be on PATH: npm install -g @altimateai/datamate. The VS Code extension already warns users when stdio mode is configured but no global datamate is found.

Related PRs

  • AltimateAI/altimate-mcp-engine#211 — writes mcpServerUrl to altimate.json (interim fix, superseded by this approach)
  • AltimateAI/vscode-altimate-mcp-server#360 — same interim fix for the VS Code extension

Requested by @saravmajestic via harness


Summary by cubic

Route datamate tools through the local MCP engine when an IDE MCP entry exists, auto-detecting stdio vs HTTP from IDE configs and reusing the IDE’s process. Consolidates tools behind a single datamate gateway, adds live reload, and keeps config and enabled state in sync across restarts.

  • Bug Fixes

    • datamate.ts: detect transport from .vscode/mcp.json, .cursor/mcp.json, .github/copilot/mcp.json; reuse IDE command; use server name datamate when an IDE entry exists, otherwise fall back to per-datamate cloud config; persist enabled: true; reconnect existing entries with MCP.connect to preserve enabled state; avoids duplicate stdio processes.
    • altimate/datamate-transport.ts: new helpers (DATAMATE_KEY, readDatamateTransportFromIde, syncDatamateUrlFromVscodeMcp, readMcpEntryFromDisk); sync the datamate entry by updatedAt (covers stdio + HTTP); URL-compare for other remote entries.
    • Local engine uses ~/.altimate/connections.json and cwd-matched sidecars to route to the right VS Code bridge; supports multiple windows.
    • mcp-discover.ts: strip ALTIMATE_EXTENSION_RPC before persisting to avoid stale/dead sockets.
    • serve.ts + config.ts: sync datamate from IDE MCP config on startup; persist updatedAt in config.
    • server.ts: add POST /altimate/mcp/reload-datamate; bypass stale in-memory config by reading fresh entries from disk and reconnect with MCP.add (no restart, no enabled flag clobber).
    • session/prompt.ts: subscribe to MCP.ToolsChanged and refresh tools on the next turn with trace logs.
    • mcp/index.ts: persist enabled/disabled to config on connect/disconnect so it survives restarts.
    • command/index.ts: add /mcps command to list MCP servers and status (AI-6948).
  • Migration

    • Ensure the datamate CLI is on PATH: npm install -g @altimateai/datamate.

Written for commit 520143f. Summary will update on new commits.

Review in cubic

@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

This PR doesn't fully meet our contributing guidelines and PR template.

What needs to be fixed:

  • PR description is missing required template sections. Please use the PR template.

Please edit this PR description to address the above within 2 hours, or it will be automatically closed.

If you believe this was flagged incorrectly, please let a maintainer know.

@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

Thanks for your contribution!

This PR doesn't have a linked issue. All PRs must reference an existing issue.

Please:

  1. Open an issue describing the bug/feature (if one doesn't exist)
  2. Add Fixes #<number> or Closes #<number> to this PR description

See CONTRIBUTING.md for details.

@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

@altimate-harness-bot

Copy link
Copy Markdown
Contributor Author

E2E Verification — PASSED ✓

Full pipeline tested end-to-end in the sandbox:

Test setup:

  • altimate serve running on port 4097 (bun source, live-reloads PR branch changes)
  • Mock Altimate backend on port 5001 (returns Jira datamate with type: "tool")
  • Mock Jira REST API on port 5001 (serves /rest/api/2/issue/AI-6348)
  • connections.json using type: "server" + jiraBaseUrl: "http://localhost:5001" (avoids need for real credentials)

Results:

✓ Initialized: AltimateMCPStdioServer
✓ Tools: 8 total, 8 Jira tools
   - jira_get_issue, jira_create_issue, jira_get_child_issues,
     jira_add_comment, jira_update_issue_status, jira_get_available_transitions,
     jira_link_issue, jira_get_link_types

→ Calling jira_get_issue(issueIdOrKey: "AI-6348")...

✓ jira_get_issue("AI-6348") SUCCESS!
  key: AI-6348
  summary: feat: Ingest Snowflake TASK_HISTORY / SERVERLESS_TASK_HISTORY / COMPLETE_TASK_GRAPHS...
  status: Done
  assignee: Raghwendra Singh

What was verified:

  1. datamate.ts patch: handleAdd() creates { type: "local", command: ["datamate", "start-stdio", "--datamate", id] } instead of cloud HTTP config
  2. mcp-discover.ts patch: stripSessionEnv() removes ALTIMATE_EXTENSION_RPC before persisting to altimate-code.json — verified environment key absent in written config
  3. Full MCP JSON-RPC stdio handshake: initialize → tools/list (8 tools) → tools/call → real tool response
  4. discoverExternalMcp reads .vscode/mcp.json correctly, transforms stdio entry to { type: "local" }, strips session socket path

@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

15 similar comments
@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

- Add stdio transport support for datamate MCP (vs HTTP-only before)
- Single-gateway mode: when .vscode/mcp.json has "datamate" key, always
  use it as server name — prevents duplicate tool sets from extension
- syncDatamateUrlFromVscodeMcp: use updatedAt field as change signal for
  the "datamate" entry (works for both stdio and HTTP), URL comparison
  for all other remote entries
- Strip ALTIMATE_EXTENSION_RPC from persisted mcp-discover configs to
  avoid stale socket paths across VS Code sessions
- persistMcpEnabled: write enabled/disabled flag to disk on MCP
  connect/disconnect so it survives session restarts
- Add /altimate/mcp/reload-datamate endpoint to re-sync and reconnect
  without full server restart
- MCP.ToolsChanged subscription in prompt loop for traceability
- Merge main: preserve trace consumer in serve.ts, restore exports on
  isAnthropicLikeModel and insertReminders
@altimate-harness-bot altimate-harness-bot Bot force-pushed the fix/datamate-stdio-local-transport branch from a71092d to 0b6cc26 Compare June 9, 2026 02:46
@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

1 similar comment
@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

@github-actions

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

1 similar comment
@github-actions

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

projectRootDir: string,
): Promise<{ type: "remote"; url: string } | { type: "local" } | null> {
try {
const mcpJsonPath = path.join(projectRootDir, ".vscode", "mcp.json")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will not be in .vscode always, it will differ according to the IDE used. We should scan across projectRootDir

"global config is at ~/.config/altimate-code/altimate-code.json. " +
"Datamate server names are prefixed with 'datamate-'. " +
"When a VS Code extension datamate entry exists (.vscode/mcp.json has 'datamate' key), " +
"'add' always uses the server name 'datamate' — tools are then prefixed 'datamate_'. " +

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is referred as add here?

for (const [key, entry] of Object.entries(serversMap)) {
const args = Array.isArray(entry["args"]) ? (entry["args"] as string[]) : []
const isDatamate =
key === "datamate" ||

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can use EXTENSION_DATAMATE_SERVER instead of string?

: transport?.type === "local"
// Extension stdio: no --datamate id needed — active teammate is resolved
// by the extension over the ALTIMATE_EXTENSION_RPC socket at runtime.
? { type: "local" as const, command: ["datamate", "start-stdio"] }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isnt this supposed to be same as in .vscode/mcp.json, so that it can re use the same process? Otherwise we will be maintaining 2 processes?

// Stripping it forces runtime discovery via ~/.altimate/extension-rpc/ sidecars,
// which always resolves the correct live bridge by matching process.cwd() against
// each bridge's recorded workspaceFolders.
function stripSessionEnv(cfg: import("../../config/config").Config.Mcp): import("../../config/config").Config.Mcp {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideal case is to use same process. May be we can expose a cleanup function, which can be called by extension on deactivate or window close? Or actually, when window is closed, this altimate code process will also be killed right?

Comment thread packages/opencode/src/cli/cmd/serve.ts Outdated
import { subscribeTraceConsumer } from "../../altimate/observability/trace-consumer"
// altimate_change end

// altimate_change start

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this logic could better be in separate file

Comment thread packages/opencode/src/cli/cmd/serve.ts Outdated
// All other remote MCP entries fall back to URL comparison (original behaviour).
// Fire-and-forget: errors are logged but never thrown.
// Returns the list of MCP server names whose config was updated.
const DATAMATE_KEY = "datamate"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

repeated constant. lets extract to common constant

Comment thread packages/opencode/src/cli/cmd/serve.ts Outdated
export async function syncDatamateUrlFromVscodeMcp(cwd: string): Promise<string[]> {
const updated: string[] = []
try {
const mcpJsonPath = path.join(cwd, ".vscode", "mcp.json")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems to be repeated logic

// check would miss the gateway-emitted specific names (#888 J1). The api.id
// checks are lowercased and tightened to a `claude-` / `anthropic-` /
// `anthropic/...` shape so a model named `foo-claude-bench` doesn't false-match.
//

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this removed? same for below comment block

@github-actions

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

1 similar comment
@github-actions

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant